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DESCRIPTION 

DIGITAL STILL CAMERA AND IMAGE CORRECTION METHOD 



Technical Field 

5 The present invention relates to a digital still camera 

and an image correction method. 
Background Art 

As seen from the fact that the total volume of 
shipments of digital still cameras exceeds that of film 

10 cameras, the digital still cameras have increasingly been in 
widespread use. Along this popularization, various levels 
of users, from high-end users to low-end users, use the 
digital still cameras and, therefore, the application of the 
digital still cameras, the required level of the image 

15 quality, the preference in the image quality, and so on have 
become diversified . 

For example, users who have used film cameras and are 
familiar with the image quality of the film cameras tend to 
prefer "f ilm-camera-like" images when the users compare 

20 images captured by digital still cameras with pictures taken 
by film cameras. In contrast, users who are alienated from 
images taken by film cameras or still pictures and are 
familiar to the image quality on TV screens tend to prefer 
"television- like" image qualities when the users compare 

25 images on TV screens with images captured by digital still 
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cameras . 

Hence, the manufacturers of the digital still cameras 
are devising images for every camera type and are developing 
the image qualities with which assumed target users seem to 
5 be satisfied. 

As described above, as the user group of digital still 
cameras becomes large, the preference in the image qualities 
is diversified. It is considerably difficult in the current 
state to realize the image qualities satisfying the 

10 preferences of the users in one digital still camera. This 
is because color images captured by digital still cameras 
are subjected to AWB control (automatic white balance 
control) , tone correction, saturation correction, and so on 
based on the knowhow unique to each of the manufacturers of 

15 the digital still cameras and, then, are recorded in flash 
memories . 

A format compliant with DCF is generally used as an 
image format when image data is recorded in a flash memory. 
An sRGB color space, which is a standard color space, is 
20 adopted in such a format. In the sRGB color space, tone 

characteristics or color characteristics including a color 
gamut (a color reproduction range) of the CRT monitors of 
personal computers are defined. 

In addition, JPEG2000, which is adopted in the DCF and 
25 which is an extension of JPEG format, and color spaces that 
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can widely cover the color reproduction range perceived by 
human beings are considered as image formats. Among these 
formats or color spaces, an scRGB color space (scene- 
reference color space) in which the amount of light of a 
5 real scene can be stored as linear data, and/or a color 
space (output-reference color space) , for example, an 
extended sRGB color space, in which the color of data is 
corrected so as to be appropriate for a destination or a 
display device, have been internationally standardized. 

10 However, whichever image format is selected, since 

captured color images are recorded and stored in a flash 
memory as a simple image in the image format compliant with 
the DCF, the images satisfying all the image qualities 
required by various users cannot be stored. 

15 There is another problem in that a captured image 

becomes a so-called failed image due to an error in setting 
the basic conditions in the image capture. For example, the 
color balance of an image can be derogated or the image can 
be underexposed because of the image capture in a natural 

20 light mode below fluorescent light. However, since digital 
image data is captured in digital still cameras, unlike the 
images taken by film cameras, there is more need for 
modification after the image capture . 

Countermeasures against the above problems include a 

25 method of utilizing commercially available "image correction 
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and processing software" in a personal computer. If a user 
owns not only a digital still camera but also a personal 
computer, images captured by the digital still camera can be 
corrected by using such software to produce appropriate 
5 images . 

However, the image data which a user can use in the 
correction is actually data subjected to JPEG compression or 
the like in the digital still camera. Since the amount of 
information in the image data is smaller than that of an 

10 actual scene, the image cannot necessarily be corrected to 

the image quality with which the user, particularly, a high- 
end user, is satisfied. 

It is difficult for and troublesome to a low-end user 
to correct the quality of a captured image to the image 

15 quality for which he/she has a preference by the use of the 
correction and processing software. Accordingly, almost all 
the commercially available correction and processing 
software has an "image-quality automatic correction 
function", and even a user who has no knowledge of the image 

20 correction can simply correct the image. However, under the 
current conditions, the correction effect can be varied 
depending on the content of the image and, therefore, a 
desired image quality cannot often be achieved. 

Furthermore, there is a problem of storage in a current 

25 sRGB format or a JPEG-YCC format compliant with the current 
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sRGB format in the correction after the image capture. As 
for the method of storing the image in a scene-reference 
color space format, such as an scRGB color space or an scYCC 
color space, a solution is proposed in, for example, 
5 Japanese Unexamined Patent Application Publication No. 2001- 
343753. However, there is currently no image correction 
software accommodated to the image in the scene-reference 
color space format . 

In order to resolve the problems described above, the 
10 present invention provides an image correction method in 

which a user can correct a captured image to an appropriate 
or desired image and which can be performed in a digital 
still camera. 



15 Disclosure of Invention 

The present invention provides, for example, a digital 
still camera including a memory that stores the image data 
of an image for which tone correction is to be performed in 
a standard color space format and a tone correction circuit . 

2 0 The image data is read out from the memory, and the tone 
correction is performed for the readout image data in the 
tone correction circuit. 

With this structure, the image is corrected to an image 
desirable to a user in the digital still camera. 



25 
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Brief Description of the Drawings 

Fig. 1 is a schematic diagram showing an embodiment of 
a capturing system according to the present invention. 

Fig. 2 is a schematic diagram showing an embodiment of 
a monitoring system according to the present invention. 

Fig. 3 is a schematic diagram showing an embodiment of 
a main part in the monitoring system. 

Fig. 4 is a schematic diagram showing an embodiment of 
a main part in the monitoring system. 

Fig. 5 includes diagrams illustrating examples of GUI 
operations in the monitoring system. 

Fig. 6 includes diagrams illustrating examples of other 
GUI operations in the monitoring system. 

Fig. 7 is a schematic diagram showing an embodiment of 
a main part in the monitoring system. 

Fig. 8 is a graph showing a characteristic of a main 
part in the monitoring system. 

Fig. 9 illustrates categories into which captured 
images are categorized. 

Fig. 10 illustrates an algorithm of a main part in the 
monitoring system . 

Fig. 11 illustrates an algorithm of a main part in the 
monitoring system . 

Fig. 12 is a graph showing a characteristic of the main 
part in the monitoring system. 
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Fig. 13 includes graphs showing characteristics of a 
main part in the monitoring system. 

Fig. 14 is a graph showing characteristics of a main 
part in the monitoring system. 
5 Fig. 15 is a table showing parameters, provided in the 

monitoring system . 

Fig. 16 illustrates an algorithm of main parts in the 
monitoring system . 

Fig. 17 illustrates an algorithm of main parts in the 
10 monitoring system. 

Fig. 18 is a graph showing characteristics of a main 
part in the monitoring system. 

Fig. 19 illustrates an algorithm of main parts in the 
monitoring system . 
15 Fig. 20 illustrates an algorithm of main parts in the 

monitoring system . 

Fig. 21 includes mathematical expressions. 

Fig. 22 includes mathematical expressions. 

Fig. 23 includes mathematical expressions. 
20 Fig. 24 includes mathematical expressions. 

Fig. 25 includes mathematical expressions. 

Fig. 26 includes mathematical expressions. 

Best Mode for Carrying Out the Invention 



25 
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(1. Signal Processing in Image Capture) 

Fig. 1 shows an example of the image capturing system 
in a 1CCD digital still camera according to the present 
invention. Specifically, an image of an object OBJ is 
5 projected on a CCD image sensor 11 through an imaging lens 
LNS. Sixteen-bit linear image data Rl , Gl , and Bl 
corresponding to three primary colors is extracted from the 
CCD image sensor 11. The extracted image data Rl/ Gl , and 
Bl is supplied to a demosaic processing circuit 12 to be 

10 demosaiced into image data for every pixel. The image data 
is then subjected to AWB processing in an AWB processing 
circuit 13 to provide image data R'2, G'2, and B'2. 

The image data R f 2, G'2, and B'2 is supplied to an 
scRGB data generating circuit 14 to provide 16 -bit linear 

15 image data RscRGB, GscRGB, and BscRGB in an scRGB format, 

and the image data RscRGB, GscRGB, and BscRGB is temporarily 
stored in a RAM 15. The stored image data RscRGB, GscRGB, 
and BscRGB is supplied to a gamma correction circuit 16 to 
be subjected to gamma correction and to provide eight -bit 

20 image data R, G, and B for the three colors. The image data 
R, G, and B is written in nonvolatile memory means, for 
example, a flash memory 20, through a reader-writer 19 and 
is stored in the flash memory 20. The flash memory 20 is 
removable from the digital still camera. 

25 Alternatively, the image data R, G, and B is supplied 
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from the gamma correction circuit 16 to a matrix calculating 
circuit 17 to be converted into image data in a YCC format, 
that is, luminance data Y and color difference data Cb and 
Cr of blue and red. The image data Y, Cb, and Cr is written 
5 in the flash memory 20 through the reader-writer 19 and is 
stored in the flash memory 20. 

The image data R, G, and B or the image data Y, Cb, and 
Cr is stored in the flash memory 20 in order to establish 
the compatibility with a known digital still camera or 

10 "image correction and processing software". According to 
the present invention, the image data is further processed 
in the following manner. 

The image data RscRGB, GscRGB, and BscRGB stored in the 
RAM 15, that is, the 16-bit linear image data, is written in 

15 the flash memory 20 through the reader-writer 19 without any 
modification and is stored in the flash memory 20. 

Alternatively, the image data RscRGB, GscRGB, and 
BscRGB is supplied to an scYCC/scRGB converting circuit 18 
to be converted into 12 -bit nonlinear image data YscYCC, 

2 0 CbscYCC, and CrscYCC in an scYCC format. The image data 

YscYCC, CbscYCC, and CrscYCC is written in the flash memory 
20 through the reader- writer 19 and is stored in the flash 
memory 20. 

The signal processing in image capture is performed in 
2 5 the manner described above. 
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(2. Example of scRGB- format Image Data Generating Circuit) 

In the scRGB data generating circuit 14, the image data 
R'2, G'2, and B f 2, which is subjected to the AWB correction, 
5 is converted into the 16 -bit linear image data RscRGB, 

GscRGB, and BscRGB in the scRGB format. This conversion is 
realized in, for example, the following process. 

(1) First, a white level (100% white level) of a scene is 
determined. For example, image data Xraw, Yraw, and Xraw is 

10 calculated from the image data R'2, G'2, and B'2, which is 

subjected to the AWB correction, for every pixel according 

to Equation 1 in Fig. 21. 

Next , the average Yraw_ave of the image data Yraw for 

all the pixels in the image or for a pixel appropriately 
15 sampled is calculated, and the white level is set to a value, 

for example, five times larger than the average Yraw_ave . 

(2) The image data Xraw, Yraw, and Xraw for every pixel is 
normalized according to Equation 2 in Fig. 21 based on the 
white level set in (1) to calculate normalized image data 

2 0 Xraw_n, Yraw_n, and Xraw_n. 

(3) As shown by Equation 3 in Fig. 21, the normalized image 
data Xraw_n, Yraw_n, and Xraw_n calculated in (2) is 
integrated with the inverse matrix of a matrix Ml to 
calculate image data R'3, G f 3, and B'3 for every pixel. 

25 (4) Sixteen-bit linear image data RscRGB, GscRGB, and BscRGB 
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in the scRGB format for every pixel is generated from the 
image data R f 3, G'3, and B'3 calculated in (3), according to 
Equation 4 in Fig. 21. 

Since the image data RscRGB , GscRGB , and BscRGB is 
5 desired image data in the scRGB format, the image data 

RscRGB, GscRGB, and BscRGB is read out from the scRGB data 
generating circuit 14 and is stored in the RAM 15. 

(3. Example of scYCC/scRGB Converting Circuit 18) 
10 In the scYCC/scRGB converting circuit 18, the image 

data RscRGB, GscRGB, and BscRGB in the scRGB format is 
converted into the image data in the scYCC format. This 
conversion is realized by, for example, the following 
process . 

15 (1) Nonlinear image data R 1 scRGB , G'scRGB, and B ■ scRGB in 

the scRGB format is calculated from the 16 -bit linear image 
data RscRGB, GscRGB, and BscRGB in the scRGB format 
according to Equation 5 in Fig. 21 and Equation 6 in Fig. 22. 
(2) The nonlinear image data R 1 scRGB , G'scRGB, and B'scRGB 

20 in the scRGB format is converted into the image data YscYCC, 
CbscYCC, and CrscYCC in the scYCC format according to 
Equations 7 and 8 in Fig. 22. 

Since the image data YscYCC, CbscYCC, and CrscYCC is 
desired image data in the scYCC format, the image data 

25 YscYCC, CbscYCC, and CrscYCC is stored in the flash memory 
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20 through the reader-writer 19. 

(4. Signal Processing in Monitoring of Captured Image) 

Fig. 2 shows an example in which the present invention 
5 is applied to a circuit for correcting a captured image. 

Specifically, the image data RscRGB, GscRGB, and BscRGB in 
the scRGB format is read out from the flash memory 20 
through the reader-writer 19, and the readout image data 
RscRGB to BscRGB is written in the RAM 15. Alternatively, 
10 the image data YscYCC, CbscYCC, and CrscYCC in the scYCC 
format is read out from the flash memory 2 0 through the 
reader-writer 19. The image data YscYCC, CbscYCC, and 
CrscYCC is supplied to the scYCC/scRGB converting circuit 18 
to be converted into the image data RscRGB, GscRGB, and 
15 BscRGB in the scRGB format. The image data RscRGB, GscRGB, 
and BscRGB is written in the RAM 15. 

The image data RscRGB, GscRGB, and BscRGB written in 
the RAM 15 is supplied to a display, for example, an LCD 32, 
through a monitor display processing circuit 31 and is 
20 displayed as a color image. 

The white balance of the image data stored in the flash 
memory 20 has been automatically controlled by the AWB 
processing circuit 13 in the image capture. In the example 
in Fig. 2, the image data written in the RAM 15 is processed 
25 in an AWB fine tuning circuit 33 to fine-tune the white 
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balance of the color image displayed in the LCD 32. 

The image data RscRGB, GscRGB , and BscRGB written in 
the RAM 15 is converted into the 12 -bit nonlinear image data 
YscYCC, CbscYCC, and CrscYCC in the scYCC format in the 
5 scYCC/scRGB converting circuit 18 and is written in a RAM 34 . 
The image data YscYCC, CbscYCC, and CrscYCC in the RAM 34 is 
corrected in accordance with a GUI operation by a user in a 
tone correction circuit 35. The corrected image data is 
converted into signals in an RGB format in the monitor 

10 display processing circuit 31, and the converted signals are 
supplied to the LCD 32 and are displayed as a color image. 

The image data that is subjected to the tone correction 
in the tone correction circuit 35 is written in the flash 
memory 2 0 through the reader-writer 19 and is stored in the 

15 flash memory 20. 

(5. Example of AWB Fine Tuning Circuit 33) 

The white balance of the image data RscRGB, GscRGB, and 
BscRGB in the scRGB format, written in the RAM 15, is fine- 
20 tuned based on the white information in the image capture, 
stored in the digital still camera. This fine tuning is 
realized by the following process. 

5-1 AT [K] correction of color temperature from white 
25 point in image capture (refer to Fig. 3) 
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5-1-1 Fine tuning of white balance on site after image 
capture 

(1) A white-point color- temperature evaluation value Tw[K] 
of a scene is yielded from the data in the image capture. A 

5 white-point color- temperature evaluation value Tw' [K] of the 
scene after the white balance is corrected is calculated 
from the evaluation value Tw [K] according to Equation 9 in 
Fig. 22 in a white-point color- temperature evaluator 331. 

(2) Xy chromaticities xd 1 and yd' are calculated from the 
10 white-point color- temperature information Tw ' [K] calculated 

in (1), according to Equation 10 in Fig. 23. It is known 
that Equation 10 is approximately satisfied at a color 
temperature of 4,000 K to 7,000 K. 

(3) XYZ tristimulus values Xw ' , Yw', and ZW are calculated 
15 from the xy chromaticities xd' and yd 1 of the white point, 

calculated in (2), according to Equation 11 in Fig. 23. 

(4) Linear RGB values R'w, G'w, and B'w in use for 
displaying in an sRGB monitor are calculated from the 
tristimulus values Xw' , Yw' , and Zw 1 of the white point 

20 after the correction by using the inverse matrix of the 

matrix Ml used in Equation (1) , according to Equation 12 in 
Fig. 23. 

(5) With regard to the white point of the Tw[K] before the 
correction, XYZ tristimulus values Xw, Yw, and Zw can be 

25 calculated according to Equations 10 and 11. 
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(6) Linear RGB values Rw, Gw, and Bw when the white point of 
the scene before the correction is displayed in the sRGB 
monitor are calculated according to Equation 12 . 

(7) White-balance control factors kr, kg, and kb are 

5 calculated from the linear RGB values Rw, Gw, and Bw before 
the correction and the linear RGB values R'w, G'w, and B'w 
after the correction according to Equation 13 in Fig. 23 in 
a white -balance control -factor calculator 332 7 

(8) In a white-balance control calculator 333, the white- 

10 balance control factors kr, kg, and kb calculated in (7) are 
integrated with the image data RscRGB, GscRGB, and BscRGB in 
the scRGB format, read out from the RAM 15, according to 
Equation 14 in Fig. 23, to calculate image data RscRGB__W, 
GscRGB_W, and BscRGB__W in the scRGB format after the 

15 correction. The calculated image data RscRGB_W, GscRGB_W, 
and BscRGB__W is written back in the RAM 15 as fine-tuned 
results . 

5-1-2 Fine tuning of white balance of image recorded 

20 in flash memory 20 

(1) White-point information in the image capture, recorded 
in advance in a header of the captured data (data scRGB or 
scYCC) , is read out to yield the white-point color 
temperature Tw[K] in the image capture. The white-balance 

25 control factors kr, kg, and kb are calculated based on this 
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information. The white-balance control factors kr, kg, and 
kb are calculated in the same manner as in 5-1-1. 

(2) When the image data stored in the flash memory 20 is the 
16 -bit linear image data RscRGB, GscRGB, and BscRGB in the 

5 scRGB format, the correction is performed in the same manner 
as in 5-1-1 (6) . 

(3) When image data stored in the flash memory 2 0 is the 12- 
bit nonlinear image data YscYCC, CbscYCC, and CrscYCC in the" 
scYCC format, the image data YscYCC, CbscYCC, and CrscYCC is 

10 converted into the 16 -bit linear image data RscRGB, GscRGB, 
and BscRGB in the scRGB format in the scYCC/scRGB converting 
circuit 18, and the correction is performed in the same 
manner as in 5-1-1 (6) . 

The conversion method is described in detail next. 

15 (3)-l The 12-bit nonlinear image data YscYCC, CbscYCC, and 
CrscYCC in the scYCC format, read out from the flash memory 
20, is converted into the nonlinear image data R'scRGB, 
G'scRGB, and B 1 scRGB in the scRGB format according to 
Equations 15 and 16 in Fig. 23. 

20 (3) -2 The image data R'scRGB, G'scRGB, and B'scRGB converted 
in (3)-l is converted into the linear image data R ! 3, G'3, 
and B'3 in the scRGB format according to Equation 17 in Fig. 
24 . 

(3) -3 The image data R'3, G'3, and B'3 converted in (3) -2 is 
25 converted into 16 -bit linear image data RscRGB, GscRGB, and 
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BscRGB in the scRGB format according to Equation 4 . 

5-2 Example of monitor display processing circuit 31 
(refer to Fig. 4) 
5 The image data RscRGB, GscRGB, and BscRGB in the scRGB 

format before and after the white balance control is 
converted into image data in the RGB format in an sc RGB /RGB 
conversion processor 311 in the monitor display processing 
circuit 31, and the converted image data is supplied to the 
10 LCD 32 and is displayed as a color image. The conversion 
from the scRGB format to the RGB format is described in 
detail next. 

(1) The 16-bit linear image data RscRGB, GscRGB, and BscRGB 
in the scRGB format is converted into the nonlinear image 

15 data R 1 scRGB , G 1 scRGB , and B ' scRGB in the scRGB format 
according to Equations 5 and 6 . 

(2) The image data converted in (1) is converted into the 8- 
bit nonlinear data R, G, and B in the RGB format according 
to Equation 18 in Fig. 25. 

20 (3) The 8-bit nonlinear image data R, G, and B converted in 
(2) is supplied to the LCD 32. 



5-3 Example of GUI for fine-tuning AWB 

An operation example of a GUI for fine-tuning the AWB 
25 is described with reference to Fig. 5. Fig. 5 includes 



- 18 - 

S03P1083 

diagrams showing the rear face of the digital still camera 
described above. On the rear face of the digital still 
camera, the LCD 32, a menu button 41, an OK button 42, and a 
cursor button 43 for up-down and left-right movement are 
5 provided . 

(1) An image for which the white-balance fine tuning is 
performed is selected and the selected image is displayed in 
the LCD 32. 

(2) The menu button 41, the cursor button 43, and the OK 
10 button 42 are operated to select a "white-balance fine 

tuning mode" . 

For example , as shown in Fig . 5A, letters indicating 
"Fine tuning", "High", "Standard", and "Low" modes are 
superimposed and displayed on the image displayed in (1) . 

15 Among these modes, which can be selected with the cursor 
button 43 and the OK button 42, the "Fine tuning" is 
provided for correction described below. When the "High" is 
selected, the white-point color temperature of the image is 
corrected to a value higher than the current value by 500 

20 [K] to 1,000 [k] . When the "Low" is selected, the white- 
point color temperature of the image is corrected to a value 
lower than the current value by 500 [K] to 1,000 [K] . When 
the "Standard" is selected, the white-point color 
temperature of the image is kept without correction. 

25 (3) When the "Fine tuning" is selected, a slide bar 44 is 
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further displayed in the LCD 32, as shown in Fig. 5B . When 
the cursor button 43 is operated to horizontally move the 
slide bar 44, for example, the white-point color temperature 
of the image is corrected to a value higher or lower than 
5 the current value by around 100 [K] each time the slide bar 
44 is moved by one scale division. 

(6. Example of Tone Correction Method in Tone Correction 
Circuit 35 (refer to Fig. 2)) 
10 The tone of 12 -bit nonlinear image data YscYCC, CbscYCC, 

and CrscYCC in the scYCC format, stored in the RAM 34, is 
corrected in the tone correction circuit 35. As described 
in detail below, the tone correction is performed in 
accordance with GUI operations by the user. 

15 

6-1 Automatic tone correction 

Fig. 6 includes diagrams illustrating examples of GUI 
operations in the tone correction. 

(1) An image whose tone is to be corrected is selected and 
20 the selected image is displayed in the LCD 32. 

(2) The menu button 41, the cursor button 43, and the OK 
button 42 are operated to select a "tone control mode" . 

For example, as shown in Fig. 6A, letters indicating 
"Manual", "Automatic", "TV", and "Picture" modes are 
25 superimposed and displayed on the image displayed in (1) . 
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Among these modes, which can be selected with the cursor 
button 43 and the OK button 42, the "Manual" is provided for 
the user to manually perform the correction described below. 
When the "TV" is selected, the image is corrected to achieve 
5 television- like image quality. When the "Picture" is 

selected, the image is corrected to achieve film-camera- like 
image quality. 

(3) When the "manual" is selected, slide bars 45 and 46 are 
further displayed in the LCD 32, as shown in Fig. 6B. When 

10 the cursor button 43 is operated to horizontally move the 
slide bar 45, the contrast of a highlight of the image is 
corrected to a value higher or lower than the current value 
by one step each time the slide bar 45 is moved by one scale 
division. When slide bar 46 is horizontally moved, the 

15 contrast of a shadow of the image is corrected to a value 

higher or lower than the current value by one step each time 
the slide bar 46 is moved by one scale division. 

(4) When the "Automatic" is selected in (2) described above, 
the automatic tone correction is performed. The automatic 

20 correction includes, as shown in Fig. 6C, a "Standard" mode 
for general automatic correction, a "Nightscape" mode for 
the automatic correction for a nightscape, and a "Snowscape" 
mode for the automatic correction for a snowscape. The user 
can select any of the "Standard", "Nightscape", and the 

2 5 " Snowscape " modes . 
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6-2 Example of tone correction circuit 35 
Fig. 7 shows an example of the tone correction circuit 
35. The tone correction circuit 35 is schematically 
5 described here, and the components in the tone correction 
circuit 35 will be described in detail below. Referring to 
Fig. 7, among the image data YscYCC, CbscYCC, and CrscYCC 
stored in the RAM 34, the luminance data YscYCC is supplied 
to a tone corrector 351. The tone of the luminance data 

10 YscYCC is corrected in the tone corrector 351 and the 

corrected data is output. The color difference data CbscYCC 
and CrscYCC is supplied to a saturation corrector 352 . The 
saturation of the color difference data CbscYCC and CrscYCC 
is corrected in the saturation corrector 352 and the 

15 corrected data is output. The output image data YscYCC, 
CbscYCC, and CrscYCC is supplied to LCD 32, as described 
above, and is displayed as a color image. The output image 
data YscYCC, CbscYCC, and CrscYCC is also supplied to the 
flash memory 20 and is stored in the flash memory 20. 

20 The luminance data YscYCC is sequentially supplied to a 

luminance signal histogram calculator 353, an image 
information extractor 354, and an image categorizer 355, and 
the images are categorized into, for example, ten categories. 
Based on the categorization result, the correction 

25 characteristics of the tone in the tone corrector 351 are 
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determined in a black-and-white-level correction curve 
generator 356, a tone correction curve generator 357, and a 
tone correction parameter selector 367. In addition, based 
on the categorization result described above, the correction 
5 characteristics of the saturation in the saturation 

corrector 352 are determined in a saturation correction 
curve generator 358 and a saturation correction parameter 
selector 368. Various parameters and thresholds are 
provided in a ROM 369. The following processing is 
10 performed in components in the tone correction circuit 35. 

6-3-1 Calculation for making cumulative histogram of 
luminance signal Y 

The luminance data YscYCC of the image for which the 
15 tone correction is to be performed is supplied from the RAM 
34 to the luminance signal histogram calculator 353. The 
luminance signal histogram calculator 353 makes the 
cumulative histogram f (Y) of luminance signals Y from the 
luminance data YscYCC, as shown in Fig. 8. 

20 

6-3-2 Extraction of image information 
The cumulative histogram f (Y) of the luminance signal Y, 
made in the luminance signal histogram calculator 353, is 
supplied to the image information extractor 3 54. The image 
25 information extractor 354 calculates values Yl, Y2 , ... Yn of 
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the luminance signal Y when the data of the cumulative 
histogram f (Y) corresponds to pl%, p2%, ... pn% (for example, 
5%, 10%, ... 95%) of the entire data, as shown in Fig. 8. The 
values Yl to Yn represent the brightness of the image. 

5 

6-3-3 Categorization of image 

The image categorizer 355 uses the image information Yl 
to Yn generated in the image information extractor 354 to 
categorize the images into 12 categories shown in Fig. 9. 
10 The image categorizer 355 performs the categorization in 

accordance with, for example, an algorithm shown in fig. 10. 

(1) The luminance signal information value Yl is compared 
with two thresholds AveLiml and AveL»im2 , which is set in 
advance, to categorize the brightness of the image into 

15 three categories; that is, "bright (Hi)", "average (Ave)", 
and "dark (Lo) » . 

(2) A value R (=Y3-Y2) is calculated from the luminance 
signal information values Y2 and Y3 . The value R is 
compared with two thresholds RangeLiml and RangeLim2 , which 

20 is set in advance, to categorize the range of the brightness 
of the image into three categories; that is, "narrow 
(Narrow)", "middle (Mid)", and "wide (Wide)". Hence, the 
image is categorized as any of a total of the nine 
categories; that is, the three categories with respect to 

25 the brightness of the image in (1) x the three categories 
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with respect to the range of the brightness of the image. 

(3) An image having a "U-shaped" histogram, shown in the 
right side in Fig. 9, is extracted from the images in the 
three categories (bright, average, and dark) belonging to 

5 the "wide (Wide) " category with respect to the range of the 
brightness. This extraction is performed by comparing the 
luminance signal values Y4 and Y5 of the image and gradient 
values SI and S2 of the shadow and the highlight in the 
cumulative histogram with thresholds U-Liml, U-Lim2, Slpl, 
10 and Slp2, which are set in advance, respectively. 

(4) The image to be corrected is categorized as any of the 
ten categories including the U-shaped histogram in the 
manner described above . 

(5) When the user selects a mode, such as the nightscape 
15 mode or the snowscape mode, in the image capture, or when 

there are inputs with the GUI in the correction of the image 
(refer to Fig. 7), two categories of "Nightscape" and 
"Snowscape" are added, as shown in the right side in Fig. 9, 
based on the image capture information or based on the 
20 inputs with the GUI, respectively. In this case, the image 
is categorized as any of a total of 12 categories. 



25 



6-3-4 Black-and-white level correction 

6-3-4-1 General black-and-white level correction 

The image data of the image categorized in the image 
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categorizer 355 is supplied to the black-and-white-level 
correction curve generator 356, as shown in Fig. 7. The 
black-and-white- level correction curve generator 356 
primarily enhances an insufficient contrast caused by the 
5 exposure state in the image capture. Accordingly, the 

black-and-white-level correction curve generator 356 has an 
algorithm, for example, shown in Fig. 11 and has an S-shaped 
characteristic shown in Fig. 12. 

The generation of this S-shaped characteristic uses 

10 functions in Equation 19 (refer to Fig. 25), in which an 
inflection point xO and a curvature rr are used as 
parameters such that a luminance value Ymin of the black 
level of the image gets close to zero and a luminance value 
Ymax of the white level of the image gets close to 1.0. The 

15 inflection point xO and the curvature rr approximate a 

broken line given by drawing a straight line between the 
luminance value Ymin and the luminance value Ymax. The 
luminance values Ymin and Ymax ordinarily correspond to both 
ends of the histogram and are determined from the luminance 

20 signal value Y, which is a value appropriately extracted 
from the values in the cumulative histogram. However, a 
threshold YTH appropriate for preventing overcorrection is 
set for the level Ymin at the black side. 

25 6-3-4-2 Black-and-white level correction of 
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"Nightscape" and "Snowscape" 

As described above in 6-3-3, a correction effect 
different from that of a category determined only from the 
cumulative histogram must be achieved in the "Nightscape" 
5 mode and the "Snowscape" mode. Hence, special processing is 
performed to correct the black-and-white level in a user 
scene selection corrector (refer to Fig. 11) . The 
information concerning the "Nightscape" mode and the 
"Snowscape" mode is yielded from the information input with 
10 the GUI by the user (refer to Fig. 7) or from the header 

information in the image file. The processing is described 
next with reference to Fig. 13. 



6-3-4-2-1 White correction of category "Nightscape" 
15 The histogram of the nightscape is characterized by 

deviating to lower tones. It is often the case that the 
histogram of the nightscape has, for example, street light 
having a relatively small area and includes objects having 
higher luminance values. In this case, as shown in the 
20 histogram of the nightscape in Fig. 9, a certain number of 
pixels are distributed near the maximum value of the tone. 
Accordingly, a sufficient correction effect cannot be 
achieved at the white side by using a method of determining 
the luminance value Ymax in normal black-and-white 
25 correction. 
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In order to perform the correction effective for such a 
high- luminance area, the white correction level for the 
nightscape is set to a value slightly smaller than the 
luminance value Ymax set by a normal method. This setting 
5 shifts a light emitter, for example, a high- luminance area 
including street light, toward the brighter area, thus 
effectively enhancing the luminance of the high- luminance 
area . 

Without the setting described above, since the 
10 luminance value detected from the cumulative histogram is 
decreased, the amount of correction to make the luminance 
value close to the white value (1.0) becomes too large. In 
contrast, with the setting described above, since the value 
at the midpoint between the detected luminance value and the 
15 white value (1.0) is used as the white correction level Ymax, 
it is possible to prevent the amount of correction from 
becoming too large. In addition, the darkness which the 
nightscape originally has is not derogated. 

20 6-3-4-2-2 Black correction of category "Nightscape" 

Since the tone of the nightscape is distributed in a 
lower range, it is difficult to obtain the effect of the 
black correction itself by using the black level Ymin 
yielded by a normal method, and the effect of the black 

25 correction level Ymin is not indispensable. However, it is 
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supposed that the tone of the shadow can be slightly 
increased due to the effect of the amount of correction at 
the white side. It is not preferable to increase the tone 
of the shadow in a wider area, as in the nightscape, because 
5 the increased tone enhances noise in the image capture 
characteristics of current digital still cameras. 

Hence, in order to surely suppress such enhancement of 
noise in the nightscape and aggressively enhance the 
darkness of the nightscape, the black correction level Ymin 
10 is fixed to a value lower than the value Ymin yielded by a 
normal method to surely darken an area, having lower tones, 
in the shadow. 



6-3-4-2-3 Black correction of category "Snowscape" 
15 Since the pixels are deviated to higher tones in the 

snowscape, contrary to the nightscape, it is difficult to 
obtain the effect of the white correction itself. However, 
it is supposed that the tone of the highlight can be 
decreased due to the effect of the amount of correction at 
20 the black side and the decreased tone of the highlight 

results in, for example, unnatural blackness such as a stain 
or a pseudo color in the highlight. 

Hence, in the snowscape, the black correction level 
Ymin is set to an appropriate value between the black level 
2 5 value Ymin yielded by a normal method and the value (0) of 



- 29 - 

S03P1083 

black to inhibit the effect of the black correction. Even 
when the contrast is enhanced due to this black correction, 
the whiteness characteristic of the snowscape is not 
derogated . 

6-3-5 Tone correction parameter selector 367 and tone 
correction 

Objects of the tone correction here is to relatively 
enhance the contrast of a range in which the tone is 
derogated mainly because of inappropriate exposure and to 
return any excess effect of the black-and-white level 
correction to a level appropriate for the category, as in 
the white level or black level correction described above. 
For example, an inverted S- shaped curve shown in Fig. 14 is 
applied to such tone correction. Functions in Equation 20 
(refer to Fig. 25) , having the inflection point xO and the 
curvature rr as the parameters, are used to generate the 
inverted S-shaped curve. 

Accordingly, the ROM 369 in Fig. 7 is provided with, 
for example, a table shown in Fig. 15 including the 
parameters of the inflection point xO and the curvature rr 
and saturation parameters kc described below for all of the 
twelve categories. The parameter xO has values in a range 
from 0.4 to 0.8. The parameter rr has values in a range 
from 1.0 to 10. 
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As shown in Fig. 16, the tone correction parameter 
selector 367 selects the corresponding parameter based on 
the category information on the image output from the image 
categorizer 355 with reference to the table in Fig. 15. The 
5 tone correction curve generator 357 generates the inverted 
S-shaped curve for the tone correction, as shown in Fig. 14, 
by using the parameters selected in the tone correction 
parameter selector 367. 

Furthermore, the tone correction parameter selector 367 

10 combines the inverted S-shaped curve for the tone correction 
with the S-shaped curve (Fig. 12) for black-and-white 
correction, generated in the black-and-white-level 
correction curve generator 356, to generate a correction 
translation table of the luminance signal values. The tone 

15 corrector 351 uses the correction translation table 

generated in the tone correction parameter selector 367 to 
convert the image data on the luminance read out from the 
RAM 34 from a value Yin to a value Yout, as shown in Fig. 14, 
and to output the converted value Yout. 

20 

6-3-6 Saturation correction 

In a YCC color space, the saturation in a range from a 
middle-saturation area to a high-saturation area can be 
derogated when the tone correction is performed for the 
25 luminance signal channel as described above. Accordingly, 
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correction for keeping the saturation is performed, along 
with the tone correction. This saturation correction is 
performed for a chroma value C yielded from the color 
difference data Cb and Cr. Basically, a gain coefficient kc 
5 of the color difference data Cb and Cr is controlled 
according to Equation 21 in Fig. 25 to enhance the 
saturation. 

Accordingly, the ROM 369 in Fig. 7 is provided with, 
for example, a table shown in Fig. 15 including the gain 

10 coefficients kc for all the twelve categories. The 
parameter kc has values in a range from 1.0 to 2.0. 

As shown in Fig. 17, the saturation correction 
parameter selector 368 selects the corresponding parameter 
kc based on the category information on the image output 

15 from the image categorizer 355 with reference to the table 
in Fig. 15. The saturation correction curve generator 358 
generates a correction curve, shown by a solid line in Fig. 
18, based on the straight line according to Equation 21 by 
using the parameter selected in the saturation correction 

20 parameter selector 368. 

In this case, in order to avoid coloring a low- 
saturation area having an almost achromatic color, an 
appropriate threshold is set for the correction curve in Fig. 
18, and the S- shaped functions according to Equation 19 are 

25 used to inhibit the saturation. An Hermite curve is used in 
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a high- saturation area in order not to clip values amplified 
in the saturation enhancement. A correction translation 
table of the saturation data Cb and Cr is generated based on 
this saturation correction curve. 
5 The saturation corrector 352 uses the saturation 

correction table generated in the saturation correction 

curve generator 358 to correct the saturation data Cb and Cr, 

read out from the RAM 34, and to output the corrected data. 

10 6-4 Tone correction and saturation correction by user 

selection 

A case in which the user selects a "TV" correction mode 
or a "picture" correction mode by operating the GUI shown in 
Fig. 6 is described next. 

15 As shown in Figs. 19 and 20, the black-and-white level 

correction by using the S-shaped curve, the tone correction 
by using the inverted S-shaped curve, and the saturation 
enhancement correction are performed in these modes, as in 
the automatic tone correction in 6-1. In these corrections, 

20 the amount of the black-and-white level correction by using 
the S-shaped curve and the amount of the saturation 
correction by using the gain coefficient are fine-tuned 
based on the amount of correction in the normal automatic 
correction . 



25 
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6-4-1 Image quality control in "TV" mode 
Images on TV screens (or images on CRT monitors) 
generally have higher average luminance, higher contrast (a 
sufficient black level and a sufficient white level) , and 
5 higher saturation. Accordingly, in order to obtain 

television- like images as a result of the image quality 
control in the "TV" mode, the correction is performed in 
consideration of these characteristics. 

10 6-4-1-1 Black-and-white level correction 

The black level correction is inhibited by comparing 
the black-and-white level correction in the "Automatic" mode 
described in 6-3-4-1. Accordingly, a black level Ymin_TV in 
this mode is set according to Equation 22 in Fig. 25 using 

15 the black level Ymin determined in the normal automatic 

correction. BKtv has a value in a range from 0.7 to 1.0. A 
white level Ymax__TV in this mode is set according to 
Equation 23 in Fig. 26 using the white level Ymax determined 
in the normal automatic correction. Wtv has a value in a 

2 0 range from 0.8 to 1.0. 

After the processing described above is performed, as 
in the generation of the black-and-white level correction 
curve in the "Automatic" correction (Fig. 12) , the 
inflection point xO and the curvature rr of the S-shaped 

25 functions (Equation 19) , which approximate a broken line 
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given by drawing a straight line between the black level 
Ymin_TV and the white level Ymax_TV, are calculated in an S- 
shaped parameter calculator (refer to Fig. 19) . 

The image having a higher average luminance and a 
5 higher contrast is captured after the correction in the 
manner described above. 

6-4-1-2 Tone correction 

As in the tone correction in the "Automatic" mode 
10 described in 6-3-5, the inverted S-shaped curve is generated 
based on the categorized information on the image, the 
inverted S-shaped curve is combined with the black-and-white 
level correction curve generated in 6-4-1-1, and the 
luminance data Y is corrected based on the correction curve 
15 resulting from the combination. 

6-4-1-3 Saturation correction 

The saturation is further enhanced in the image quality 
control in the "TV" mode, compared with the correction based 

20 on the categorized information on the image in the 

saturation correction in the "Automatic" mode described in 
6-3-6. Accordingly, a gain coefficient kc_TV for the 
saturation correction is calculated from the gain 
coefficient kc set based on the categorized information on 

25 the image, according to Equation 24 in Fig. 26. Gtv has a 
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value in a range from 1.0 to 1.2. 

The processing described above is performed in a user 
selection corrector in the saturation correction curve 
generator 358 in Fig. 20. Then, the correction curve is 
5 generated in the same manner as in the generation of the 
saturation correction curve in the "Automatic" mode. 

6-4-2 Image quality control in "Picture" mode 
Picture images generally have higher contrast, but have 
10 lower average luminance than that of the images on the TV 

screens. Accordingly, in order to achieve film-camera- like 
image quality as a result of the correction in the "Picture" 
mode, the correction is performed in consideration of these 
characteristics . 

15 

6-4-2-1 Black-and-white level correction 
The black level is determined in the same manner as in 
the black-and-white level correction in the "Automatic" mode 
described in 6-3-4-1. A white level Ymax_Pic is calculated 
20 according to Equation 25 in Fig. 26 using the white level 
Ymax determined in the "Automatic" mode. Wpic has a value 
in a range from 0.8 to 1.0. 

After the processing described above is performed, as 
in the generation of the black-and-white level correction 
25 curve in the "Automatic" correction (Fig. 12), the 
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inflection point xO and the curvature rr of the S- shaped 
functions (Equation 19) , which approximate a broken line 
given by drawing a straight line between the black level 
Ymin and the white level Ymax_Pic, are calculated in the S- 
shaped parameter calculator (refer to Fig. 19). 

The image having a higher contrast and maintaining the 
halftones is captured after the correction in the manner 
described above . 

6-4-2-2 Tone correction 

The tone correction is performed in the same manner as 
in the tone correction in the "TV" mode described in 6-4-1-2. 

6-4-2-3 Saturation correction 

The saturation correction is basically performed in the 
same manner as in the saturation correction described in 6- 
4-1-3. The saturation is further enhanced in the image 
quality control in the "Picture" mode, compared with the 
correction based on the categorized information on the image 
in the saturation correction in the "Automatic" mode 
described in 6-3-6. Accordingly, a gain coefficient kc_pic 
for the saturation correction is calculated from the gain 
coefficient kc set based on the categorized information on 
the image, according to Equation 26 in Fig. 26. Gpic has a 
value in a range from 1.0 to 1.2. 
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The processing described above is performed in the user 
selection corrector in the saturation correction curve 
generator 358 in Fig. 20. Then, the correction curve is 
generated in the same manner as in the generation of the 
saturation correction curve in the "Automatic" mode. 

6-4-3 Tone control mode by user 

When the user selects the "Manual" in the GUI operation 
shown in Fig. 6, the user can control the contrast of the 
highlight in the image and the contrast of the shadow in the 
image, as shown in Fig. 6B. 

6-4-3-1 Black-and-white level correction 
As shown in Fig. 6B, a black level Ymin_User of the 
image is corrected by using the slide bar 46 for controlling 
the contrast of the shadow. In this case, the black level 
Ymin_JJser is calculated according to Equation 27 in Fig. 26 
using the black level Ymin in the black-and-white level 
correction in the "Automatic" mode described in 6-3-4-1. 
BKuser has a value in range from 0.85 to 1.15. That is, the 
black level can be corrected, by using the slide bar 46, 
from 0.85 (the minimum contrast of the shadow) to 1.15 (the 
maximum contrast of the shadow) in increments of 0.05. 

A white level value Ymax_User of the image is corrected 
by using the slide bar 45 for controlling the contrast of 
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the highlight. The white level Ymax_User is calculated 
according to Equation 28 in Fig. 26 using the white level 
Ymax in the black-and-white level correction in the 
"Automatic" mode described in 6-3-4-1. Wuser has a value in 
5 range from 0.85 to 1.15 (however, when Ymax_User exceeds 1.0, 
Ymax_User = 1.0). That is, the white level can be corrected, 
by using the slide bar 45, from 0.85 (the maximum contrast 
of the highlight) to 1.15 (the minimum contrast of the 
highlight) in increments of 0.05. 

10 After the processing described above is performed, as 

in the generation of the black-and-white level correction 
curve in the "Automatic" correction (Fig. 12), the 
inflection point xO and the curvature rr of the S- shaped 
functions (Equation 19) , which approximate a broken line 

15 given by drawing a straight line between the black level 

Ymin_User and the white level Ymax_User, are calculated in 
the S-shaped parameter calculator (refer to Fig. 19) . 

6-4-3-2 Tone correction 
2 0 The tone correction is performed in the same manner as 

in the tone correction described in 6-4-1-2. 



25 



6-4-3-3 Saturation correction 

The amount of the saturation correction is determined 
in accordance with the amount of the black-and-white level 
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correction calculated in 6-4-3-1. Accordingly, a gain 
coefficient kc__User for the saturation correction by the 
user is calculated from the gain coefficient kc set based on 
the categorized information on the image, according to 
5 Equation 29 in Fig. 26. Guser has a value in a range from 
0.85 to 1.15. The value of Guser is determined according to 
Equation 30 in Fig. 26 using the values of BKuser and Wuwer, 
which vary with the control by the use of the slide bars 45 
and 26 for the highlight and shadow in the black-and-white 

10 level correction. 

The processing described above is performed in the user 
selection corrector in the saturation correction curve 
generator 358 in Fig. 20. Then, the correction curve is 
generated in the same manner as in the generation of the 

15 saturation correction curve in the "Automatic" mode. 

6-5 Monitoring of image after tone correction (refer 
to Fig. 4) 

The images corrected in the manners described above are 
20 converted into 8 -bit nonlinear signals in the RGB format in 
a YcbCr/RGB conversion processor 312 in the monitor display 
processing circuit 31, and are supplied to the LCD 32 and 
displayed as the images. The conversion in the YcbCr/RGB 
conversion processor is performed according to a matrix 
25 operation in Equation 31 in Fig. 26. M3-1 is the inverse 
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matrix of the matrix used in Equation 7. 

(7. Features of digital still camera described above) 

(1) Since the RAM 15, which stores the image data in the 
5 scene-reference color space format, that is, the 16-bit 

linear image data in the scRGB format in the above 
description, is provided in the digital still camera, the 
user can control the white balance of the image only with 
the digital still camera on site after the image capture 
10 without using a personal computer or "image correction and 
processing software 11 . 

(2) Similarly, the user can correct the tone and saturation 
of the image only with the digital still camera on site 
after the image capture. 

15 (3) Even when the image is captured by another digital still 
camera, copying the image data in the flash memory 2 0 allows 
the white balance of the image to be controlled owing to the 
provision of the RAM 15 . 

(4) Similarly, even when the image is captured by another 
20 digital still camera, copying the image data in the flash 

memory 2 0 allows the tone and saturation of the image to be 
corrected . 

(5) Since the captured image is corrected based on the 
statistical analysis of the image when the tone and 

25 saturation of the image is automatically corrected, the 
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qualities of various captured images can be improved with 
higher probability. 

(6) The correction curve generated by combining the S-shaped 
function and the inverted S-shaped function is used in the 

5 tone correction, so that the highlight of the image can be 
corrected independent of the shadow of the image to some 
extent . 

(7) Even a low-end user who does not have sufficient knowhow 
for correcting the tone or saturation of the captured image 

10 can automatically correct a failed image with simple GUI 
operations . 

(8) High- end user who has a certain amount of knowhow for 
correcting the tone or saturation of the captured image can 
also simply correct the image to his/her taste with GUI 

15 operations. 

(9) Television-like images or film-camera- like images 
according to the preference of the user can be produced by 
the correction with simple GUI operations. 

2 0 (8. Others) 

When the CCD image sensor 11 includes three CCD image 
sensors corresponding to the three primary colors in the 
digital still camera described above, the demosaic 
processing circuit 12 is not necessary. The flash memory 20 

25 may be a removable memory card, such as a memory stick 
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10 



15 



(registered trademark) . Furthermore, the image data stored 
in the flash memory 20 may be output to an external device, 
such as a personal computer or a printer, through a USB or 
the like. 

[List of Abbreviation Used in This Description] 
AWB : Auto White Balance 
CCD: Charge Coupled Device 
CRT: Cathode Ray Tube 

DCF: Design rule for Camera File Format 

GUI : Graphical User Interface 

JPEG: Joint Photographic Experts Group 

LCD: Liquid Crystal Display 

RAM: Random Access Memory 

ROM: Read Only Memory 

scRGB : relative SCene RGB color space 

TV: Television 

USB: Universal Serial Bus 



Industrial Applicability 

2 0 According to the present invention, since the memory, 

which stores the image data in the scene-reference color 
space format, is provided in the digital still camera, the 
user can control the white balance of the image and/or can 
correct the tone and saturation of the image, only with the 

25 digital still camera on site after the image capture. In 
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addition, even when the image is captured by another digital 
still camera, the white balance, tone, and saturation of the 
image can be corrected. 

Since the tone and saturation is automatically 
corrected based on the statistical analysis of the image, 
the qualities of various captured images can be improved 
with higher probability. The correction curve generated by 
combining the S-shaped function and the inverted S- shaped 
function is used in the tone correction, so that the 
highlight of the image can be corrected independent of the 
shadow of the image to some extent . 

Even a low-end user who does not have sufficient 
knowhow for correcting the tone or saturation can 
automatically correct the tone or saturation of a failed 
image with simple GUI operations. Furthermore, a high-end 
user who has a certain amount of knowhow for correcting the 
tone or saturation can also correct the image to his/her 
taste with GUI operations. Television- like images or film- 
camera-like images according to the preference of the user 
can be produced by the correction with simple GUI operations. 



